Technical Field
The present disclosure relates to a method and apparatus of detecting hand gestures with respect to a display apparatus, and the execution of processing based on the detected gesture.
Description of Related Art
Conventionally, a mouse, a pen, a touch panel, etc., may be referred to as pointing devices with which a user may perform input operations with respect to an electronic device that includes a display screen. These input elements are suitable when a user is relatively close to the display screen. For example, a stylus or finger may be used to make direct contact with a touch panel display implemented in a handheld device such as a smartphone.
On the other hand, for display devices having a relatively large display screen, such as a television receiver, pointing devices are difficult to use because a user typically operates the relatively large devices from a position away from the display screen. For such relatively large display devices, wireless devices such as remote controllers that transmit infrared signals are typically utilized for control of the display devices.
However, operation by a remote controller greatly differs from an intuitive user operation such as the touch operation, which is currently common in smartphones and the like. For example, a user must search for a button on the remote control corresponding to the desired operation rather than performing an intuitive motion that causes the desired operation to be executed. Moreover, it is troublesome to keep a remote controller in hand, and a user may often lose the remote controller and therefore be unable to control the device remotely.
In recent years, a three-dimensional (3D) gesture detection technique of realizing a pointer function by detecting a 3D motion of a user's hand and/or finger has been proposed. A motion detection apparatus implementing the aforementioned 3D gesture detection technique may include a distance image sensor, which may include an image sensor (e.g., CMOS sensor) and infrared ray light emitting diode (LED).
In 3D gesture detection, although it is anticipated that intuitive operations can be performed, the action corresponding to clicking with a mouse and a scroll function becomes an important control element. In conventional 3D gesture detection techniques, for example, the change of the position of a hand of a direction perpendicular to a screen may be detected by the distance image sensor, and the detected gesture may be determined to correspond to a clicking action (e.g., as in a click performed with a mouse).
However, when a user tries to perform the foregoing clicking action by pressing the hand toward the display screen, it is difficult to make the gesture action completely to the depth direction of the display screen. That is, the action which presses a hand into the specific axial direction of a sensor will be unavoidably accompanied by movement of the hand with respect to another axial direction of the sensor. As a result, a motion of a hand in a direction (the x-axis or y-axis direction, relative to the axis perpendicular to the screen) parallel to a display screen is detected, and this detected motion is determined to correspond to a scroll operation. However, the scroll operation may not have been desired when the user performed the action corresponding to the click operation. Therefore, the user's intended operation cannot be detected correctly. In fact, it is often incorrectly detected as a different input operation.
Moreover, a gesture action in the depth direction of a display screen places a heavier physical burden on the user compared with vertical and horizontal actions because the user must utilize more joints to perform the gesture.
Regarding a scroll function, scroll actions may be detected in conventional techniques by tracking a movement of a target object (e.g., a user's hands) in arbitrary directions relative to the display. However, when the target object reaches the boundaries of the detectable range of the sensors, a reverse direction scroll is detected when the target object is returned back to original direction, which is not user friendly.